Data

Sources

We are collecting our Data from the sustainability development report (SDG), the international labour organization (ILOSTAT), the World Bank, Our world in data, the CATO institute, one from Kaggle (disasters: we couldn’t find relevant accessible information from somewhere else) and GitHub. We found different datasets containing useful information in relation with the SDGs. The details about these data and the links are presented in the next question.

Name of the Table Source
D1_1_SDG https://dashboards.sdgindex.org/
downloads
D2_2_Unemployment_rate https://www.ilo.org/
shinyapps/bulkexplorer3/
?lang=en&segment=indicator
&id=UNE_2EAP_SEX_AGE_RT_A
D3_0_GDP_per_capita https://data.worldbank.org/
indicator/NY.GDP.PCAP.CD
D3_1_Military_expenditure_percent_GDP https://data.worldbank.org/
indicator/MS.MIL.XPND.GD.ZS
D3_2_Military_expenditure_percent_gov_exp https://data.worldbank.org/
indicator/MS.MIL.XPND.ZS
D4_0_Internet_usage https://ourworldindata.org/
grapher/share-of-individuals-
using-the-internet
D5_0_Human_freedom_index https://www.cato.org/
human-freedom-index/2022
D6_0_Disaters https://www.kaggle.com/
datasets/brsdincer/all-natural-
disasters-19002021-eosdis
D7_0_COVID https://github.com/
owid/covid-19-data/tree/master/public/data
D8_0_Conflicts https://datacatalog.worldbank.org/
search/dataset/0041070/
Global-Spread-of-Conflict-by-
Country-and-Population

Description

During the wrangling process: we add data to our table (D1_1_SDG) based on different other datasets and match them based on the country, the country code, and the year. The table below shows all our 9 databases that we merge to have our final table for the analysis, as well as each variable of interest that we keep.

Table
Name
Variable
Name
Explanation # obs before cleaning # obs
after cleaning
In all database code Country code (ISO)
country Name of the country
year Year of the observation (2000-2022)
D1_1_
SDG
overallscore Overall score on all 17 SDGs (the score are % of achievement of the goals determined by the UN based on several indicators) 4140 observations of 120 variables 3818 observations of 21 variables
goal1:goal17 Score on each SDG except SDG 14 (16 variables)
population Number of people living in the country
D2_2_
Unemployment_rate
unemployment.rate Unemployment rate (% of the population 15 years old and older) 82800 observations of 8 variables 571 observations of 5 variables
D3_0_
GDP_per_capita
GDPpercapita GDP per capita 266 observations of 68 variables 3818 observations of 4 variables
D3_1_
Military_expenditure_
percent_GDP
MilitaryExpenditure
PercentGDP
Military expenditures in percentage of GDP 266 observations of 68 variables 3818 observations of 4 variables
D3_2_
Military_expenditure_
percent_gov_exp
MilitaryExpenditure
PercentGovExp
Military expenditures in percentage of government expenditures 266 observations of 68 variables 3818 observations of 4 variables
D4_0_
Internet_usage
internet.usage Internet usage (% of the population) 6570 observations of 4 variables 3433 observations of 4 variables
D5_0_
Human_freedom_index
region Part of the world, group of countries (e.g. Eastern Europe, Dub-Saharan Africa, South Asia, etc.) 3465 observations of 141 variables 3339 observations of 18 variables
hf_score Human Freedom score = mean of personal freedom (PF) and economic freedom (EF).
pf_law

Rule of law, mean score of:

  • Procedural justice
  • Civil justice
  • Criminal justice
  • Rule of law (V-Dem)
pf_security

Security and safety, mean score of:

  • Homicide
  • Disappearances conflicts,
  • terrorism
pf_movement

Freedom of movement (V-Dem)

Freedom of movement (CLD)

pf_religion

Freedom of religion

Religious organizatio repression

pf_assembly

Civil society entry and exit

Freedom of assembly

Freedom to form/run political parties

Civil society repression

pf_expression

Direct attacks on the press

Media and expression (V-Dem)

Media and expression (Freedom House)

Media and expression (BTI)

Media and expression (CLD)

pf_identity

Same-sex relationships

Divorce

Inheritance rights

Female genital mutilation

pf_score Mean of every PF component score
ef_government

Government consumption

Transfers and subsidies

Government investment

Top marginal tax rate

State ownership of assets

ef_legal

Judicial independence

Impartial courts

Protection of property rights

Military interference Integrity of the legal system Legal enforcementof contracts

Regulatory costs

Reliability of police

ef_money

Money growth

Standard deviation of inflation

Inflation: Most recent year

Freedom to own foreign currency

ef_trade

Tariffs

Regulatory trade barriers

Black-market exchange rates

Movement of capital and people

ef_regulation

Credit market regulations

Labor market regulations

Business regulations

ef_score Mean of every EF component score
D6_0_
Disaters
continent Continents touched by the disasters such as floods, ouragan 14644 observations of 47 variables

2435 observations

of 10 variables

total_deaths Number of total deaths caused by the disasters
no_injured Number of injured people
no_affected Number of affected people
no_homeless Number of people that lost their home and are now homeless
total_affected Sum of people affected (sum of the variables: no_injured, no-affected, no_homeless)
total_damages Total of infrastructure damages
D7_0_
COVID
deaths_per_million Number of people dead due to COVID 349966 observations of 67 variables 501 observations (only between 2020-2022, before no COVID) of 6 variables
cases_per_million Number of COVID cases
stringency Government Response Stringency Index: composite measure based on 9 response indicators including school closures, workplace closures, and trave
D8_0_
Conflicts
ongoing Variable coded 1 for more than 25 deaths in intrastate conflict and 0 otherwise according to UCDP/PRIO Armed Conflict Dataset 17.1. 5016 observations of 18 variables 2782 observations of 8 variables
sum_deaths Best estimate of deaths in all categories of violence (non-state, one-sided and state-based) recorded by the Uppsala Conflict Data Program in the country based on the UCDP GED dataset (unpublished 2016 data). The location of these events is used for estimating the extent of violence.
pop_affected Share of population affected by violence in percentage (0 to 100) measured as described above based on population data from CIESIN, the PRIO-GRID structure as well as UCDP GED.
area_affected Area affected by conflict
maxintensity Two different intensity levels are coded: minor armed conflicts (1) and wars (2), Takes the max intensity of conflict in the country so that it is coded 2 if there is at least one war (>=1000 deaths in intrastate conflict) during the year. Data from UCDP/PRIO Armed Conflict Dataset 17.1.

Wrangling/cleaning

To accommodate the large scale of the datasets we intended to utilize, we decided to pre-clean each of our datasets before merging them. This allowed us to simplify the process of cleaning our final dataset afterwards.

1. Dataset on SDG

This is our main dataset, that we clean in order to keep the columns containing the following information: country name, country code, year, population, overall score and the SDGs scores.

We begin by importing the data and transforming it into a dataframe. We rename the columns and transform the scores into numeric variables.

D1_0_SDG <- read.csv(here("scripts","data","SDG.csv"), sep = ";")
D1_0_SDG <- as.data.frame(D1_0_SDG)

D1_0_SDG <- D1_0_SDG[,1:22]

colnames(D1_0_SDG) <- c("code", "country", "year", "population", "overallscore", "goal1", "goal2", "goal3", "goal4", "goal5", "goal6", "goal7", "goal8", "goal9", "goal10", "goal11", "goal12", "goal13", "goal14", "goal15", "goal16", "goal17")

D1_0_SDG[["overallscore"]] <- as.double(gsub(",", ".", D1_0_SDG[["overallscore"]]))

makenumSDG <- function(D1_0_SDG) {
  for (i in 1:17) {
    varname <- paste("goal", i, sep = "")
    D1_0_SDG[[varname]] <- as.double(gsub(",", ".", D1_0_SDG[[varname]]))
  }
  return(D1_0_SDG)
}

D1_0_SDG <- makenumSDG(D1_0_SDG)

We continue by inspecting the missing values.

propmissing <- numeric(length(D1_0_SDG))

for (i in 1:length(D1_0_SDG)){
  proportion <- mean(is.na(D1_0_SDG[[i]]))
  propmissing[i] <- proportion
}
propmissing
#>  [1] 0.0000 0.0000 0.0000 0.0778 0.0000 0.0833 0.0000 0.0000 0.0000
#> [10] 0.0000 0.0000 0.0000 0.0000 0.0000 0.0944 0.0000 0.0000 0.0000
#> [19] 0.2278 0.0000 0.0000 0.0000

Seeing that population has a lot of NAs, we investigate and find out that it is normal to have missing values, because some of the observations are not countries but regions, so we can drop these observations.

SDG0 <- D1_0_SDG |> 
  group_by(code) |> 
  select(population) |> 
  summarize(NaPop = mean(is.na(population))) |>
  filter(NaPop != 0)
print(SDG0, n = 180)
#> # A tibble: 14 x 2
#>    code         NaPop
#>    <chr>        <dbl>
#>  1 _Africa          1
#>  2 _E_Euro_Asia     1
#>  3 _E_S_Asia        1
#>  4 _HIC             1
#>  5 _LAC             1
#>  6 _LIC             1
#>  7 _LIC_LMIC        1
#>  8 _LMIC            1
#>  9 _MENA            1
#> 10 _OECD            1
#> 11 _Oceania         1
#> 12 _SIDS            1
#> 13 _UMIC            1
#> 14 _World           1

D1_0_SDG <- D1_0_SDG %>%
  filter(!str_detect(code, "^_"))

Now there is no more missing values in the variable population and we see that we have information on 166 countries.

(country_number <- length(unique(D1_0_SDG$country)))
#> [1] 166

We see that there are only NAs in 3 SDG scores: 1, 10 and 14 and that when there are NAs for a country, it is on all years or none. We decide to run ore investigations of those 3 SDG scores to decide if we keep them or not for the analysis.

SDG1 <- D1_0_SDG |> 
  group_by(code) |> 
  select(contains("goal")) |> 
  summarize(Na1 = mean(is.na(goal1)),
            Na2 = mean(is.na(goal2)),
            Na3 = mean(is.na(goal3)),
            Na4 = mean(is.na(goal4)),
            Na5 = mean(is.na(goal5)),
            Na6 = mean(is.na(goal6)),
            Na7 = mean(is.na(goal7)),
            Na8 = mean(is.na(goal8)),
            Na9 = mean(is.na(goal9)),
            Na10 = mean(is.na(goal10)),
            Na11 = mean(is.na(goal11)),
            Na12 = mean(is.na(goal12)),
            Na13 = mean(is.na(goal13)),
            Na14 = mean(is.na(goal14)),
            Na15 = mean(is.na(goal15)),
            Na16 = mean(is.na(goal16)),
            Na17 = mean(is.na(goal17))) |>
  filter(Na1 != 0 | Na2 != 0 | Na3 != 0| Na4 != 0| Na5 != 0| Na6 != 0| Na7 != 0| Na8 != 0| Na9 != 0| Na10 != 0| Na11 != 0| Na12 != 0| Na13 != 0| Na14 != 0| Na15 != 0| Na16 != 0| Na17 != 0)

kable(for (col in names(SDG1)[-1]) {
  print(paste(col, "count:", sum(SDG1[[col]] != 0)))
})
#> [1] "Na1 count: 15"
#> [1] "Na2 count: 0"
#> [1] "Na3 count: 0"
#> [1] "Na4 count: 0"
#> [1] "Na5 count: 0"
#> [1] "Na6 count: 0"
#> [1] "Na7 count: 0"
#> [1] "Na8 count: 0"
#> [1] "Na9 count: 0"
#> [1] "Na10 count: 17"
#> [1] "Na11 count: 0"
#> [1] "Na12 count: 0"
#> [1] "Na13 count: 0"
#> [1] "Na14 count: 40"
#> [1] "Na15 count: 0"
#> [1] "Na16 count: 0"
#> [1] "Na17 count: 0"

For goal 1, there are only 9.04% missing values in 15 different countries. Goal 1 being “end poverty”, we decide to keep it and only remove the countries with no information for the analysis.

SDG2 <- D1_0_SDG |> 
  group_by(code) |> 
  select(contains("goal")) |> 
  summarize(Na1 = mean(is.na(goal1))) |>
  filter(Na1 != 0)

print(table(SDG2$Na1))
#> 
#>  1 
#> 15

length(unique(SDG2$code))/country_number
#> [1] 0.0904

For goal 10, there are only 10.2% missing values in 17 different countries. Goal 10 being “reduced inequalities”, we decide to keep it and only remove the countries with no information for the analysis.

SDG3 <- D1_0_SDG |> 
  group_by(code) |> 
  select(contains("goal")) |> 
  summarize(Na10 = mean(is.na(goal10))) |>
  filter(Na10 != 0)

print(table(SDG3$Na10))
#> 
#>  1 
#> 17

length(unique(SDG3$code))/country_number
#> [1] 0.102

For goal 14, there are 24.1% missing values in 40 different countries. Goal 14 being “life under water”, we decide not to keep it, because other SDG such as “life on earth” and “clean water” already treat similar subjects.

SDG4 <- D1_0_SDG |> 
  group_by(code) |> 
  select(contains("goal")) |> 
  summarize(Na14 = mean(is.na(goal14))) |>
  filter(Na14 != 0)

print(table(SDG4$Na14))
#> 
#>  1 
#> 40

length(unique(SDG4$code))/country_number
#> [1] 0.241

D1_0_SDG <- D1_0_SDG %>% select(-goal14)

We will be working with different datasets and merge them based on the country code and the year. To make sure the match will work well, we verify that the country names are encoded in UTF-8 format, then we standardize the name of the countries (we needed to make a custom matrch for Turkey) and the country codes using the countrycode library. In addition, we create a list of all the country codes contained in the main database in order to filter the other databases. Finally, we complete the database to make sure all the combinations of “country, year” are in the database. The number of rows isn’t changed.

D1_0_SDG$country <- stri_encode(D1_0_SDG$country, to = "UTF-8")

D1_0_SDG <- D1_0_SDG %>%
  mutate(country = countrycode(country, "country.name", "country.name", custom_match = c("T�rkiye"="Turkey")))

D1_0_SDG$code <- countrycode(
  sourcevar = D1_0_SDG$code,
  origin = "iso3c",
  destination = "iso3c",
)

list_country <- c(unique(D1_0_SDG$code))

D1_0_SDG_country_list <- D1_0_SDG %>%
  filter(code %in% list_country) %>%
  select(code, country)

D1_0_SDG_country_list <- D1_0_SDG_country_list %>%
  select(code, country) %>%
  distinct()

Finally, we complete database to make sure there are not couples of (year, code) missing.

Here are the first few lines of the cleaned dataset on SDG achievement scores:

As said, this is now our main dataset. All subsequent datasets will be merged with this dataset. Therefore, for all the following datasets, we want to make sure that we only keep data for the same countries and years as in this dataset. We have a total of 166 countries and the years range from 2000 to 2022.

2. Dataset on Unemployment rate

In this dataset, the initial step involves importing the data. Next, we ensure that the names and codes of the countries are formatted in UTF-8, preventing any discrepancies due to mismatches in country names. Following this, we modify the column names and filter the data to include only the relevant countries and years, specifically the years 2000 to 2022, covering 166 countries from our primary dataset.

D2_1_Unemployment_rate <- read.csv(here("scripts","data","UnemploymentRate.csv")) %>%
  as.data.frame() %>%
  mutate(
    country = iconv(ref_area.label, to = "UTF-8", sub = "byte"),
    country = countrycode(country, "country.name", "country.name"),
    year = time,
    `unemployment rate` = obs_value / 100,
    age_category = classif1.label,
    sex = sex.label
  ) %>%
  select(-ref_area.label, -time, -obs_value, -classif1.label, -sex.label, -source.label, -obs_status.label, -indicator.label) %>%
  merge(D1_0_SDG_country_list[, c("country", "code")], by = "country", all.x = TRUE) %>%
  filter(year >= 2000 & year <= 2022,
         !str_detect(sex, fixed("Male")) & !str_detect(sex, fixed("Female")),
         code %in% D1_0_SDG_country_list$code,
         age_category == "Age (Youth, adults): 15+") %>%
  select(code, country, year, `unemployment rate`) %>%
  distinct()

Here are the first few lines of the cleaned dataset on Unemployment rate:

3. Dataset on GDP military Expenditures

We have three different databases which contain information on each countries over the years. Each year represent one variable. We want to extract three variables for our analysis: GDP per capita, military expenditures in percentage of the GDP and military expenditures in percentage of government expenditures.

GDPpercapita <-
  read.csv(here("scripts","data","GDPpercapita.csv"), sep = ";")
MilitaryExpenditurePercentGDP <-
  read.csv(here("scripts","data","MilitaryExpenditurePercentGDP.csv"), sep = ";")
MiliratyExpenditurePercentGovExp <-
  read.csv(here("scripts","data","MiliratyExpenditurePercentGovExp.csv"), sep = ";")

After importing the data, we fill in the missing country codes using the column Indicator.Name, because we realized after some manipulations, that some of the country codes were false, but the next column contained the right ones.

fill_code <- function(data){
  data <- data %>%
    mutate(Country.Code = ifelse(!grepl("^[A-Z]{3}$", Country.Code), Indicator.Name, Country.Code))
}

We create a set of functions that we will apply to each database. First, remove the variables that we don’t need, which are the years before 2000. Second, make sure that the values are numeric and rename the year variables (because they all had an “X” before year number). Third, transform the database from wide to long, in order to match the main database. Fourth, transform the year variable into an integer variable and rearrange and rename the columns to match the ones of the other databases. Then, we apply these transformations to the three databases.

remove <- function(data){
  years <- seq(1960, 1999)
  removeyears <- paste("X", years, sep = "")
  data <- data[, !(names(data) %in% c("Indicator.Name", "Indicator.Code", "X", removeyears))]
}

makenum <- function(data) {
  for (i in 2000:2022) {
    year <- paste("X", i, sep = "")
    data[[year]] <- as.numeric(data[[year]])
  }
  return(data)
}

renameyear <- function(data) {
  for (i in 2000:2022) {
    varname <- paste("X", i, sep = "")
    names(data)[names(data) == varname] <- gsub("X", "", varname)
  }
  return(data)
}

wide2long <- function(data) {
  data <- pivot_longer(data, 
                       cols = -c("Country.Name", "Country.Code"), 
                       names_to = "year", 
                       values_to = "data")
  return(data)
}

yearint <- function(data) {
  data$year <- as.integer(data$year)
  return(data)
}

nameorder <- function(data) {
  colnames(data) <- c("country", "code", "year", "data")
  data <- data %>% select(c("code", "country", "year", "data"))
}

cleanwide2long <- function(data){
  data <- fill_code(data)
  data <- remove(data)
  data <- makenum(data)
  data <- renameyear(data)
  data <- wide2long(data)
  data <- yearint(data)
  data <- nameorder(data)
}

GDPpercapita <- cleanwide2long(GDPpercapita)
MilitaryExpenditurePercentGDP <- cleanwide2long(MilitaryExpenditurePercentGDP)
MiliratyExpenditurePercentGovExp <- cleanwide2long(MiliratyExpenditurePercentGovExp)

We rename the colums with the main information, standardize the country code and remove the countries that are not in our main database. We see that all the 166 countries are there.

GDPpercapita <- GDPpercapita %>%
  rename(GDPpercapita = data)
MilitaryExpenditurePercentGDP <- MilitaryExpenditurePercentGDP %>%
  rename(MilitaryExpenditurePercentGDP = data)
MiliratyExpenditurePercentGovExp <- MiliratyExpenditurePercentGovExp %>%
  rename(MiliratyExpenditurePercentGovExp = data)

GDPpercapita$code <- countrycode(
  sourcevar = GDPpercapita$code,
  origin = "iso3c",
  destination = "iso3c",
)

MilitaryExpenditurePercentGDP$code <- countrycode(
  sourcevar = MilitaryExpenditurePercentGDP$code,
  origin = "iso3c",
  destination = "iso3c",
)

MiliratyExpenditurePercentGovExp$code <- countrycode(
  sourcevar = MiliratyExpenditurePercentGovExp$code,
  origin = "iso3c",
  destination = "iso3c",
)

GDPpercapita <- GDPpercapita %>% filter(code %in% list_country)
length(unique(GDPpercapita$code))
#> [1] 166

MilitaryExpenditurePercentGDP <- MilitaryExpenditurePercentGDP %>% filter(code %in% list_country)
length(unique(MilitaryExpenditurePercentGDP$code))
#> [1] 166

MiliratyExpenditurePercentGovExp <- MiliratyExpenditurePercentGovExp %>% filter(code %in% list_country)
length(unique(MiliratyExpenditurePercentGovExp$code))
#> [1] 166

There were only 157 countries that were both in the main SDG dataset and in these 3 datasets, but we suspected that some of the missing countries were in the database but not rightly matched. Indeed, Bahamas was in the database but instead of the code “BHS” there was “The”, for “COD” it was “Dem. Rep.”, for “COG” it was “Rep”, etc. We remarked that the code is in another column of the initial database: “Indicator.Name”. We went back to the initial database and before cleaning it we put the right codes (as seen above) and after rerunning the code we see that we have all our 166 countries from the initial dataset.

list_country_GDP <- c(unique(GDPpercapita$code))
(missing <- setdiff(list_country, list_country_GDP))
#> character(0)

We run a first round of investigation of the missing values and find that we have 16.4% for MiliratyExpenditurePercentGovExp, 12.9% for MilitaryExpenditurePercentGDP and 1.31% for GDPpercapita.

mean(is.na(MiliratyExpenditurePercentGovExp$MiliratyExpenditurePercentGovExp))
#> [1] 0.164
mean(is.na(MilitaryExpenditurePercentGDP$MilitaryExpenditurePercentGDP))
#> [1] 0.129
mean(is.na(GDPpercapita$GDPpercapita))
#> [1] 0.0131

GDP per capita

For GDPpercapita, only two countries (SOM and SSD) have a lot of missing values and in total 11 countries countries have missing values.

GDPpercapita1 <- GDPpercapita %>%
  group_by(code) %>%
  summarize(NaGDP = mean(is.na(GDPpercapita))) %>%
  filter(NaGDP != 0)
print(GDPpercapita1, n = 180)
#> # A tibble: 11 x 2
#>    code   NaGDP
#>    <chr>  <dbl>
#>  1 AFG   0.130 
#>  2 BTN   0.0435
#>  3 CUB   0.0870
#>  4 LBN   0.0435
#>  5 SOM   0.565 
#>  6 SSD   0.652 
#>  7 STP   0.0435
#>  8 SYR   0.0870
#>  9 TKM   0.0870
#> 10 VEN   0.304 
#> 11 YEM   0.130

We plot the evolution of GDPpercapita avec the years for each country containing missing values and distinguish the percentage of missing values with colors.

filtered_data_GDP <- GDPpercapita %>%
  filter(code %in% GDPpercapita1$code) # countries with NAs

filtered_data_GDP <- filtered_data_GDP %>%
  group_by(code) %>%
  mutate(PercentageMissing = mean(is.na(GDPpercapita))) %>% # column % NAs
  ungroup()

Evol_Missing_GDP <- ggplot(data = filtered_data_GDP) +
  geom_point(aes(x = year, y = GDPpercapita, 
                 color = cut(PercentageMissing,
                             breaks = c(0, 0.1, 0.2, 0.3, 1),
                             labels = c("0-10%", "10-20%", "20-30%", "30-100%")))) +
  labs(title = "Evolution of GDP per capita over time", x = "Year", y = "GDP per capita") +
  scale_color_manual(values = c("0-10%" = "blue", "10-20%" = "green", "20-30%" = "red", "30-100%" = "black"),
                     labels = c("0-10%", "10-20%", "20-30%", "30-100%")) +
  guides(color = guide_legend(title = "% missings")) +
  facet_wrap(~ code, nrow = 4)

print(Evol_Missing_GDP)

For the countries with less than 30% of missing values and a linear evolution in time, we fill the missing values using linear interpolation.

list_code <- c("AFG", "BTN", "CUB", "STP", "TKM")

for (i in list_code) {
  country_data <- GDPpercapita %>% filter(code == i)
  interpolated_data <- na.interp(country_data$GDPpercapita)
  GDPpercapita[GDPpercapita$code == i, "GDPpercapita"] <- interpolated_data
}

Military expenditures in percentage of GDP

For MilitaryExpenditurePercentGDP, 12 countries have 100% of missing values. We further investigate and keep them for now, knowing that some of these coutries may also have many missing values in the other databases when wee merge everything and will be dropped later.

MilitaryExpenditurePercentGDP1 <- MilitaryExpenditurePercentGDP %>%
  group_by(code) %>%
  summarize(NaMil1 = round(mean(is.na(MilitaryExpenditurePercentGDP)),3)) %>%
  filter(NaMil1 != 0)

print(table(MilitaryExpenditurePercentGDP1$NaMil1))
#> 
#> 0.043 0.087  0.13 0.174 0.217 0.261 0.304 0.348 0.391 0.522 0.565 
#>     4     2     7     6     3     3     3     2     1     2     2 
#> 0.739 0.783     1 
#>     1     1    12

We plot the evolution of MilitaryExpenditurePercentGDP along the years for each country containing missing values and distinguish the percentage of missing values with colors.

filtered_data_Mil1 <- MilitaryExpenditurePercentGDP %>%
  filter(code %in% MilitaryExpenditurePercentGDP1$code) # countries with NAs

filtered_data_Mil1 <- filtered_data_Mil1 %>%
  group_by(code) %>%
  mutate(PercentageMissing = mean(is.na(MilitaryExpenditurePercentGDP))) %>% # Column % NAs
  ungroup()

Evol_Missing_Mil1 <- ggplot(data = filtered_data_Mil1) +
  geom_line(aes(x = year, y = MilitaryExpenditurePercentGDP, 
                 color = cut(PercentageMissing,
                             breaks = c(0, 0.1, 0.2, 0.3, 1),
                             labels = c("0-10%", "10-20%", "20-30%", "30-100%")))) +
  labs(title = "Military expenditure in % of GDP over time", x = "Years from 2000 to 2022", y = "GDP per capita") +
  scale_color_manual(values = c("0-10%" = "blue", "10-20%" = "green", "20-30%" = "red", "30-100%" = "black"),
                     labels = c("0-10%", "10-20%", "20-30%", "50-100%")) +
  guides(color = guide_legend(title = "% missings")) +
  facet_wrap(~ code, nrow = 5) +
  theme(strip.text = element_text(size = 6)) +
  scale_x_continuous(breaks = NULL) +
  scale_y_continuous(breaks = NULL)

print(Evol_Missing_Mil1)

For the countries with less than 30% of missing values and a linear evolution in time, we fill the missing values using linear interpolation.

list_code <- c("AFG", "BDI", "BEN", "CAF", "CIV", "COD", "GAB", "GMB", "KAZ", "LBN", "LBR", "MNE", "MRT", "NER", "TKJ", "TTO", "ZMB")

for (i in list_code) {
  country_data <- MilitaryExpenditurePercentGDP %>% filter(code == i)
  interpolated_data <- na.interp(country_data$MilitaryExpenditurePercentGDP)
  MilitaryExpenditurePercentGDP[MilitaryExpenditurePercentGDP$code == i, "MilitaryExpenditurePercentGDP"] <- interpolated_data
}

Military expenditures in percentage of governement expenditures

For MilitaryExpenditurePercentGovExp, 17 countries have 100% of missing values. We further investigate and keep them for now, knowing that some of these coutries may also have many missing values in the other databases when wee merge everything and will be dropped later.

MiliratyExpenditurePercentGovExp1 <- MiliratyExpenditurePercentGovExp %>%
  group_by(code) %>%
  summarize(NaMil2 = round(mean(is.na(MiliratyExpenditurePercentGovExp)),3)) %>%
  filter(NaMil2 != 0)

print(table(MiliratyExpenditurePercentGovExp1$NaMil2))
#> 
#> 0.043 0.087  0.13 0.174 0.217 0.261 0.304 0.348 0.391 0.478 0.522 
#>     5     3     5     4     5     4     4     2     1     1     2 
#> 0.565 0.609 0.783     1 
#>     2     1     1    17

We plot the evolution of MilitaryExpenditurePercentGovExp along the years for each country containing missing values and distinguish the percentage of missing values with colors.

filtered_data_Mil2 <- MiliratyExpenditurePercentGovExp %>%
  filter(code %in% MiliratyExpenditurePercentGovExp1$code) # Countries with NAs

filtered_data_Mil2 <- filtered_data_Mil2 %>%
  group_by(code) %>%
  mutate(PercentageMissing = mean(is.na(MiliratyExpenditurePercentGovExp))) %>% # Column % NAs
  ungroup()

Evol_Missing_Mil2 <- ggplot(data = filtered_data_Mil2) +
  geom_line(aes(x = year, y = MiliratyExpenditurePercentGovExp, 
                 color = cut(PercentageMissing,
                             breaks = c(0, 0.1, 0.2, 0.3, 1),
                             labels = c("0-10%", "10-20%", "20-30%", "30-100%")))) +
  labs(title = "Military expenditure in % of government expenditures over time", x = "Year from 2000 to 2022", y = "GDP per capita") +
  scale_color_manual(values = c("0-10%" = "blue", "10-20%" = "green", "20-30%" = "red", "30-100%" = "black"),
                     labels = c("0-10%", "10-20%", "20-30%", "50-100%")) +
  guides(color = guide_legend(title = "% missings")) +
  facet_wrap(~ code, nrow = 5) +
  theme(strip.text = element_text(size = 6)) +
  scale_x_continuous(breaks = NULL) +
  scale_y_continuous(breaks = NULL)

print(Evol_Missing_Mil2)

For the countries with less than 30% of missing values and a linear evolution in time, we fill the missing values using linear interpolation.

list_code <- c("AFG", "ARM", "BEN", "BIH", "BLR", "COG", "ECU", "GAB", "GMB", "KAZ", "LBN", "LBR", "MNE", "MWI", "NER", "TTO", "UKR", "ZMB")

for (i in list_code) {
  country_data <- MiliratyExpenditurePercentGovExp %>% filter(code == i)
  interpolated_data <- na.interp(country_data$MiliratyExpenditurePercentGovExp)
  MiliratyExpenditurePercentGovExp[MiliratyExpenditurePercentGovExp$code == i, "MiliratyExpenditurePercentGovExp"] <- interpolated_data
}

We now look again at the percentage of missing values for the trhee databases: 14.49% for MiliratyExpenditurePercentGovExp, 11.6% for MilitaryExpenditurePercentGDP and 1.07% for GDPpercapita

mean(is.na(MiliratyExpenditurePercentGovExp$MiliratyExpenditurePercentGovExp))
#> [1] 0.149
mean(is.na(MilitaryExpenditurePercentGDP$MilitaryExpenditurePercentGDP))
#> [1] 0.116
mean(is.na(GDPpercapita$GDPpercapita))
#> [1] 0.0107

D3_1_GDP_per_capita <- GDPpercapita
D3_2_Military_Expenditure_Percent_GDP <- MilitaryExpenditurePercentGDP
D3_3_Miliraty_Expenditure_Percent_Gov_Exp <- MiliratyExpenditurePercentGovExp

Here are the first few lines of the cleaned dataset of GDP per capita:

Here are the first few lines of the cleaned dataset of military expenditures in percentage of GDP:

Here are the first few lines of the cleaned dataset of military expenditures in percentage of government expenditures:

4. Dataset on internet usage

To prepare the dataset on internet usage in the world to be merge with the other data, we first, import the data. Then, we keep only the year that we are interested in (2000 to 2022). We also rename the column and keep only the country that match the list of the countries in the main dataset on the SDG.

D4_0_Internet_usage <- read.csv(here("scripts", "data", "InternetUsage.csv")) %>%
  filter(Year >= 2000, Year <= 2022) %>%
  rename(
    code = Code,
    country = Entity,
    year = Year,
    internet_usage = Individuals.using.the.Internet....of.population.
  ) %>%
  mutate(internet_usage = internet_usage / 100) %>%
  filter(code %in% list_country)

Here are the first few lines of the cleaned dataset of internet usage:

5. Dataset on human freedom index

After importing the data from the CATO Institute website, we noticed that even if the file was called “Human Freedom Index 2022”, the available observations were only going from 2000 up to 2020. We have decided first to modify it in order to match our other datasets, by renaming/encoding/standardizing the columns containing the country names.

data <- read.csv(here("scripts", "data", "human-freedom-index-2022.csv"))

#data in tibble 
datatibble <- tibble(data)

# Rename the column countries into country to match the other datbases
names(datatibble)[names(datatibble) == "countries"] <- "country"

# Make sure the encoding of the country names are UTF-8
datatibble$country <- iconv(datatibble$country, to = "UTF-8", sub = "byte")

# standardize country names
datatibble <- datatibble %>%
  mutate(country = countrycode(country, "country.name", "country.name"))

Once done, we could verify which countries were or were not present between these observations and our main SDG dataset. We have decided to keep the ones that were matching between the two datasets.

# Merge by country name
datatibble <- datatibble %>%
  left_join(D1_0_SDG_country_list, by = "country")

datatibble <- datatibble %>% filter(code %in% list_country)
(length(unique(datatibble$code)))
#> [1] 159

# See which ones are missing
list_country_free <- c(unique(datatibble$code))
(missing <- setdiff(list_country, list_country_free))
#> [1] "AFG" "CUB" "MDV" "STP" "SSD" "TKM" "UZB"

# Turkey was missing but present in the initial database (it was a problem when stadardizing the country names of D1_0SDG_country_list that we corrected) and the other missing countries are:"AFG" "CUB" "MDV" "STP" "SSD" "TKM" "UZB" 
D5_0_Human_freedom_index <- datatibble

Then, we noticed that there were a lot of columns that were not important for us, as we had 141 variables taken into account. So we have decided to keep the ones that refers to the countries informations (such as code, year, ..) and their human freedom scores per category (pf for personnal freedom, ef for economical freedom).

# erasing useless columns to keep only the general ones. 
D5_0_Human_freedom_index <- select(D5_0_Human_freedom_index, year, country, region, hf_score, pf_rol, pf_ss, pf_movement, pf_religion, pf_assembly, pf_expression, pf_identity, pf_score, ef_government, ef_legal, ef_money, ef_trade, ef_regulation, ef_score, code)

D5_0_Human_freedom_index <- D5_0_Human_freedom_index %>%
  rename(
    pf_law = names(D5_0_Human_freedom_index)[5],      # Renames the 5th column to "pf_law"
    pf_security = names(D5_0_Human_freedom_index)[6]  # Renames the 6th column to "pf_security"
  )

After renaming the columns pf_law/security for comprehension purpose, we have investigated how are distributed the NA values among the countries and the variables. After having found the percentages of missing values per country and variable, heatmaps revealed themself to be a great tool for visualizing datas.

na_percentage_by_country <- D5_0_Human_freedom_index %>%
  group_by(country) %>%
  select(-code) %>%
  summarise(across(everything(), ~mean(is.na(.))*100))

na_long <- na_percentage_by_country %>%
  pivot_longer(
    cols = -country,
    names_to = "Variable",
    values_to = "NA_Percentage"
  )

overall_na_percentage <- na_long %>%
  group_by(Variable) %>%
  summarize(Avg_NA_Percentage = mean(NA_Percentage, na.rm = TRUE)) %>%
  arrange(desc(Avg_NA_Percentage))
print(overall_na_percentage)
#> # A tibble: 17 x 2
#>    Variable      Avg_NA_Percentage
#>    <chr>                     <dbl>
#>  1 ef_money                 10.4  
#>  2 ef_trade                 10.4  
#>  3 ef_score                 10.4  
#>  4 hf_score                 10.4  
#>  5 pf_score                 10.4  
#>  6 ef_regulation             9.49 
#>  7 ef_government             2.91 
#>  8 ef_legal                  1.71 
#>  9 pf_law                    1.44 
#> 10 pf_identity               0.299
#> 11 pf_assembly               0    
#> 12 pf_expression             0    
#> 13 pf_movement               0    
#> 14 pf_religion               0    
#> 15 pf_security               0    
#> 16 region                    0    
#> 17 year                      0

Then, for having a better understanding of the situation, we ordered the countries having at least 1 variable containing 50% and more of missing values

na_long <- na_long %>%
  group_by(country) %>%
  mutate(Count_NA_50_100 = sum(NA_Percentage >= 50 & NA_Percentage <= 100, na.rm = TRUE)) %>%
  ungroup() %>%
  arrange(desc(Count_NA_50_100))

heatmap_ordered_all <- ggplot(na_long, aes(x = reorder(country, -Count_NA_50_100), y = Variable)) +
  geom_tile(aes(fill = NA_Percentage), colour = "white") +
  scale_fill_gradient(low = "white", high = "red") +
  theme_minimal() +
  labs(
    title = "Heatmap of NA Percentages per Country and Variable",
    x = "Countries",
    y = "Variables",
    fill = "NA Percentage"
  ) +
  theme(
    axis.text.x = element_blank(),  # Hide x-axis labels
    axis.text.y = element_text(size = 9)
  )
print(heatmap_ordered_all)

We notice that only some countries look to contain at least 50% of missing values and in addition that most of the missing values are concerning the EF variables (Economic Freedom). Now, we tried to produce another heatmap only containing the ordered countries, and also counting for each one of these country the number of variables with at least 50% of NAs.

na_long_filtered <- na_long %>%
  group_by(country) %>%
  mutate(Count_NA_50_100 = sum(NA_Percentage >= 50 & NA_Percentage <= 100, na.rm = TRUE)) %>%
  filter(Count_NA_50_100 > 0) %>%
  ungroup() %>%
  arrange(desc(Count_NA_50_100))

heatmap_ordered_filtered <- ggplot(na_long_filtered, aes(x = reorder(country, -Count_NA_50_100), y = Variable)) +
  geom_tile(aes(fill = NA_Percentage), colour = "white") +
  scale_fill_gradient(low = "white", high = "red") +
  theme_minimal() +
  labs(
    title = "Heatmap of NA Percentages per Country and Variable",
    x = "Countries",
    y = "Variables",
    fill = "NA Percentage"
  ) +
  theme(
    axis.text.x = element_text(angle = 90, hjust = 1),
    axis.text.y = element_text(size = 7)
  )
print(heatmap_ordered_filtered)

country_na_count <- na_long %>%
  filter(NA_Percentage >= 50) %>%
  group_by(country) %>%
  summarise(Count_NA_50_100 = n()) %>%
  arrange(desc(Count_NA_50_100))
print(country_na_count)
#> # A tibble: 13 x 2
#>    country  Count_NA_50_100
#>    <chr>              <int>
#>  1 Comoros                8
#>  2 Djibouti               8
#>  3 Somalia                8
#>  4 Belarus                6
#>  5 Guinea                 6
#>  6 Iraq                   6
#>  7 Laos                   6
#>  8 Sudan                  6
#>  9 Bhutan                 5
#> 10 Liberia                5
#> 11 Bahamas                1
#> 12 Belize                 1
#> 13 Brunei                 1

We conclude here that 13 countries were concerned by our selection of 50% and more of missing values. When discussing between us, we came to the conclusion that among these 13 countries, a great part of them were not going to be selected because they had a lot of missing values in our main dataset too. Therefore, we have decided to merge this data with the other datasets and finish the cleaning after.

Here are the first few lines of the partialy cleaned dataset on Human Freedom Index scores:

6. Dataset on Disasters

For this dataset concerning the Disasters we imported the data from Kaggle as we couldn’t find the original dataset that is private coming from the EOSDIS SYSTEM, an interactive interface for browsing full-resolution, global, daily satellite images from NASA. Once we made sure that our file called “Disasters” was convert into a data frame, we selected some specific columns that we where interested in.

Disasters <- read.csv(here("scripts","data","Disasters.csv"))

Disasters <- as.data.frame(Disasters)

Disasters <- Disasters %>%
  select(Year, Country, ISO, Location, Continent, Disaster.Subgroup, Disaster.Type, Total.Deaths, No.Injured, No.Affected, No.Homeless, Total.Affected, Total.Damages...000.US..)

Because we knew that our file showed all the disasters in each country over the years (between 1970-2021) and that we wanted to focus on a specific period, we filtered our data to show the years between 2000 and 2022. Then we rearranged our data, changing the data types of all the columns and their names in order to match our other datasets.

# Rearrange the columns, changed the type of data, renamed the columns
Rearanged_Disasters <- Disasters %>%
  filter(Year >= 2000 & Year <= 2022) %>%
  mutate(
    code = as.character(ISO),
    country = as.character(Country),
    year = as.integer(Year),
    continent = as.character(Continent),
    disaster.subgroup = as.character(Disaster.Subgroup),
    disaster.type = as.character(Disaster.Type),
    location = as.character(Location),
    total.deaths = as.numeric(Total.Deaths),
    no.injured = as.numeric(No.Injured),
    no.affected = as.numeric(No.Affected),
    no.homeless = as.numeric(No.Homeless),
    total.affected = as.numeric(Total.Affected),
    total.damages = as.numeric(Total.Damages...000.US..)
  )

We then grouped the data by “year”, “code”, “country” and “continent” and summarize the data. Here you can see that we re-selected specific columns as we saw that our first pre-selection was still too wide and some variables as the disaster.subgroup and disaster.type weren’t pertinent.We arranged the columns based on “code,” “country,” “year,” and “continent” to match the other datasets.

Disasters <- Rearanged_Disasters %>%
  group_by(year,code, country, continent) %>%
  summarize(
    total_deaths = sum(total.deaths, na.rm = TRUE),
    no_injured = sum(no.injured, na.rm = TRUE),
    no_affected = sum(no.affected, na.rm = TRUE),
    no_homeless = sum(no.homeless, na.rm = TRUE),
    total_affected = sum(total.affected, na.rm = TRUE),
    total_damages = sum(total.damages, na.rm = TRUE)
  ) 

D6_0_Disasters <- Disasters %>%
  select(code, country, year, continent, total_deaths, no_injured, no_affected, no_homeless, total_affected, total_damages) %>%
  arrange(code, country, year, continent)

Finally we filtered our disasters data to keep only the countries that are present in our main dataset. We analysed the missing countries and identified three countries (BHR, BRN, MLT) that are unexpectedly missing.

D6_0_Disasters <- D6_0_Disasters %>% filter(code %in% list_country)
length(unique(D6_0_Disasters$code))
#> [1] 163

# Here we see which countries are missing
list_country_disasters <- c(unique(D6_0_Disasters$code))
(missing <- c(missing,setdiff(list_country, list_country_disasters)))
#>  [1] "AFG" "CUB" "MDV" "STP" "SSD" "TKM" "UZB" "BHR" "BRN" "MLT"

Here are the first few lines of the cleaned dataset on Disasters:

7. Dataset on COVID

This dataset contains information on the COVID19 pandemic between 2020 and 2022. The observation are by year, month, day. After importing the database, we transform the date in format YYYY-MM-DD in order to only keep the year.

COVID <- read.csv(here("scripts","data","COVID.csv"))

COVID <- COVID[,c("iso_code", "location", "date", "new_cases_per_million", "new_deaths_per_million", "stringency_index")]

COVID$date <- as.integer(year(COVID$date))

We perform a first round of investigation of the missing values before aggregating the values by year. We begin with the variables “cases per million” and “deaths per million”: seeing that for each country, we have either only missing values, either a very low percentage of missing values (~1%), we can compute the sum over each year and ignore the missing values without altering the data. Indeed, where al the values are missing, the computation will return a NA. We then look at the “stringency” variable and we have 3 scenarios:

  1. ~20% missings: we ignore missing values when computing the mean to have an idea of stringency each year (because we compute the mean stringency over the year, if some days are missing, it is not a problem, it can not evoluate that fast).

  2. all are missing : we can ignore the missing values when computing the mean, because it will still return a missing value

  3. almost all are missing: here the mean doesn’t make sense -> we will replace the values by NAs to be coherent. The countries with this issues are: ERI, GUM, PRI and VIR. We verify if they are in our main dataset and since none of these countries are, we can ignore the issue, the lines will be remove later anyway.

We aggregate the observations of all days of a year in one observation per country using the mean.

COVID1 <- COVID %>%
  group_by(iso_code) %>%
  summarize(NaCOVID = round(mean(is.na(new_cases_per_million)),3)) %>%
  filter(NaCOVID != 0)

print(table(COVID1$NaCOVID))
#> 
#> 0.001 0.002 0.003 0.004 0.012 0.109     1 
#>    33     6     2     5     1     1     9

COVID2 <- COVID %>%
  group_by(iso_code) %>%
  summarize(NaCOVID = round(mean(is.na(new_deaths_per_million)),3)) %>%
  filter(NaCOVID != 0)

print(table(COVID2$NaCOVID))
#> 
#> 0.001 0.002 0.004  0.11     1 
#>    32     1     2     1     9

COVID3 <- COVID %>%
  group_by(iso_code) %>%
  summarize(NaCOVID = round(mean(is.na(stringency_index)), 3)) %>%
  filter(NaCOVID != 0)

print(table(COVID3$NaCOVID))
#> 
#>  0.13 0.186 0.198  0.21 0.986     1 
#>     1     1     1   178     4    70

issue_list <- c("ERI", "GUM", "PRI", "VIR")
is.element(issue_list, list_country)
#> [1] FALSE FALSE FALSE FALSE

COVID <- COVID %>%
  group_by(location, date) %>%
  mutate(
    cases_per_million = sum(new_cases_per_million, na.rm = TRUE),
    deaths_per_million = sum(new_deaths_per_million, na.rm = TRUE),
    stringency = mean(stringency_index, na.rm = TRUE)
  )%>%
  ungroup()

Now that all the variables of interest are aggregated by year, we remove all the variables that we don’t need and rename all the remaining variables to match the main dataset.

COVID <- COVID %>%
  group_by(location, date) %>%
  distinct(date, .keep_all = TRUE) %>%
  ungroup()

COVID <- COVID %>% select(-c(new_cases_per_million, new_deaths_per_million, stringency_index))

colnames(COVID) <- c("code", "country", "year", "cases_per_million", "deaths_per_million", "stringency")

We remove the years that exceed 2022, we make sure that the country codes are all iso codes with 3 letters (we observe that sometimes they are preceded by “OWID_”) and we standardize the country codes.

COVID <- COVID[COVID$year <= 2022, ]

COVID$code <- gsub("OWID_", "", COVID$code)

COVID$code <- countrycode(
  sourcevar = COVID$code,
  origin = "iso3c",
  destination = "iso3c"
)

We remove the observations of countries that aren’t in our main dataset on SDGs and find that all the 166 countries that we have in the main SDG dataset are also in this one.

COVID <- COVID %>% filter(code %in% list_country)
length(unique(COVID$code))
#> [1] 166

We perform a second round of missing values investigation and find out that there are no missing values except for the stringency, where there are 4.19%. Either all values are missing for one country, or 50% are missing, so these 7 countries won’t be included when analyzing the effect of stringency on the SDG scores.

mean(is.na(COVID$cases_per_million))
#> [1] 0
mean(is.na(COVID$deaths_per_million))
#> [1] 0
mean(is.na(COVID$stringency))
#> [1] 0.0419

COVID4 <- COVID %>%
  group_by(code) %>%
  summarize(NaCOVID = mean(is.na(stringency))) %>%
  filter(NaCOVID != 0)
print(COVID4, n = 300)
#> # A tibble: 7 x 2
#>   code  NaCOVID
#>   <chr>   <dbl>
#> 1 ARM       1  
#> 2 COM       1  
#> 3 MDV       1  
#> 4 MKD       1  
#> 5 MNE       1  
#> 6 NAM       0.5
#> 7 STP       1

D7_0_COVID <- COVID

Here are the first few lines of the cleaned dataset on COVID19:

8. Dataset on Conflicts

For our conflicts dataset, we imported the data from “The World Banck” data catalog. Once we made sure that our file called “Disasters” was convert into a data frame, we selected some specific columns that we where interested in.

Conflicts <- read.csv(here("scripts","data","Conflicts.csv"))

Conflicts <- as.data.frame(Conflicts)

Conflicts <- Conflicts %>%
  select(year, country, ongoing, gwsum_bestdeaths, pop_affected, peaceyearshigh, area_affected, maxintensity, maxcumulativeintensity)

Our file showed all the Conflicts and consequences per country over the years (between 2000-2016). We couldn’t find a better and more complete dataset, As we consider conflicts as events, we will only take into account results between 2000 and 2016. Then we rearranged our data, changing the data types of all the columns and their names in order to match our other datasets. We grouped the data by ” year”, “country”, re-selected some variables and summarize the data.

Rearanged_Conflicts <- Conflicts %>%
  filter(year >= 2000 & year <= 2022)%>%
  mutate(
    ongoing = as.integer(ongoing),
    country = as.character(country),
    year = as.integer(year),
    gwsum_bestdeaths = as.numeric(gwsum_bestdeaths),
    pop_affected = as.numeric(pop_affected),
    area_affected = as.numeric(area_affected),
    maxintensity = as.numeric(maxintensity),
    )

# Group the data by "year", "country" and summarize the data
Conflicts <- Rearanged_Conflicts %>%
  group_by(year, country) %>%
  summarize(
    ongoing = sum (ongoing, na.rm = TRUE),
    sum_deaths = sum(gwsum_bestdeaths, na.rm = TRUE),
    pop_affected = sum(pop_affected, na.rm = TRUE),
    area_affected = sum(area_affected, na.rm = TRUE),
    maxintensity = sum(maxintensity, na.rm = TRUE),
  )

After we Selected specific columns from the summarized data and arrange the data by our specified columns. To make our dataset compatible with the main one and let the merging face succeed, we dd some adjustment concerning the country names’ to ensure the compatibility. Then we standardize and merge by country names to finally rearrange the data to retain only the countries present in our main dataset. Note that in the end we can see that only one country is missing that wasn’t in the initial conflicts database: BLR

conflicts <- Conflicts %>%
  select(country, year, ongoing, sum_deaths, pop_affected, area_affected, maxintensity) %>%
  arrange(country, year)

conflicts$country <- iconv(conflicts$country, to = "UTF-8", sub = "byte")

conflicts <- conflicts %>%
  mutate(country = countrycode(country, "country.name", "country.name"))

conflicts <- conflicts %>%
  left_join(D1_0_SDG_country_list, by = "country")

conflicts <- conflicts %>%
  select(code, country, year, ongoing, sum_deaths, pop_affected, area_affected, maxintensity) %>%
  arrange(code, country, year)


D8_0_Conflicts <- conflicts %>% filter(code %in% list_country)
(length(unique(conflicts$code)))
#> [1] 166

# See which countries are missing
list_country_conflicts <- c(unique(conflicts$code))
(missing <- c(missing, setdiff(list_country, list_country_conflicts)))
#>  [1] "AFG" "CUB" "MDV" "STP" "SSD" "TKM" "UZB" "BHR" "BRN" "MLT"
#> [11] "BLR"

Here are the first few lines of the cleaned dataset on Conflicts:

Merge data

By merging our eight pre-cleaned datasets, we create a final database.

D2_1_Unemployment_rate$country <- NULL
merge_1_2 <- D1_0_SDG |> left_join(D2_1_Unemployment_rate, join_by(code, year))

D3_1_GDP_per_capita$country <- NULL
merge_12_3 <- merge_1_2 |> left_join(D3_1_GDP_per_capita, join_by(code, year))

D3_2_Military_Expenditure_Percent_GDP$country <- NULL
merge_12_3 <- merge_12_3 |> left_join(D3_2_Military_Expenditure_Percent_GDP, join_by(code, year)) 

D3_3_Miliraty_Expenditure_Percent_Gov_Exp$country <- NULL
merge_12_3 <- merge_12_3 |> left_join(D3_3_Miliraty_Expenditure_Percent_Gov_Exp, join_by(code, year)) 

D4_0_Internet_usage$country <- NULL
merge_123_4 <- merge_12_3 |> left_join(D4_0_Internet_usage, join_by(code, year)) 

D5_0_Human_freedom_index$country <- NULL
merge_1234_5 <- merge_123_4 |> left_join(D5_0_Human_freedom_index, join_by(code, year)) 

D6_0_Disasters$country <- NULL
merge_12345_6 <- merge_1234_5 |> left_join(D6_0_Disasters, join_by(code, year)) 

D7_0_COVID$country <- NULL
D7_0_COVID <- D7_0_COVID |> distinct(code, year, .keep_all = TRUE)
merge_123456_7 <- merge_12345_6 |> left_join(D7_0_COVID, join_by(code, year)) 

D8_0_Conflicts$country <- NULL
all_Merge <- merge_123456_7 |> left_join(D8_0_Conflicts, join_by(code, year)) 

all_Merge <- all_Merge %>% filter(!code %in% missing)

Cleaning of the final database

We replace the NAs of the COVID columns by 0 (because we don’t have real missing, only introduced by merging for the years before COVID).

all_Merge <- all_Merge %>%
  mutate(
    cases_per_million = ifelse(is.na(cases_per_million), 0, cases_per_million),
    deaths_per_million = ifelse(is.na(deaths_per_million), 0, deaths_per_million),
    stringency = ifelse(is.na(stringency), 0, stringency)
  )

Since we took the information on the continent and region from databases that are not the main one, we complete these inforamtion for the whole final dataset.

all_Merge <- all_Merge %>%
  group_by(country) %>%
  mutate(continent = ifelse(is.na(continent), first(na.omit(continent)), continent)) %>%
  ungroup()

all_Merge <- all_Merge %>%
  group_by(country) %>%
  mutate(region = ifelse(is.na(region), first(na.omit(region)), region)) %>%
  ungroup()

We order the database, beginning by the information on the country, the year, the continent and the region.

all_Merge <- all_Merge %>%
  select(code, year, country, continent, region, everything())

write.csv(all_Merge, file = here("scripts","data","all_Merge.csv"))

Here are the first few lines of the final dataset:

Final structure of our merged database: each country of the 166 countries from D1_1_SDG are observed each year from 2000 to 2022, thus each row has a key composed of (code, year) that uniquely identifies an observation. The other columns are the variables listed above. Due to some countries having a lot of missing information we will have to eliminate some of them, but we will still have more than 2000 rows in our database.

Treatment of missing values

We load our final database and subset it according to the data that we will need in order to answer the different questions. This will help us dealing with the missing values.

all_Merge <- read.csv(here("scripts","data","all_Merge.csv"))

For question 1, we only keep the years until 2020, because most of the explanatory variables that we want to use (those coming from the human freedom index) only have values until 2020.

data_question1 <- all_Merge %>% filter(year<=2020) %>% select(-c(total_deaths, no_injured, no_affected, no_homeless, total_affected, total_damages, cases_per_million, deaths_per_million, stringency, ongoing, sum_deaths, pop_affected, area_affected, maxintensity))

For question 2 and 4, we use the main data from the SDG database.

data_question24 <- all_Merge %>% select(c(code, year, country, continent, region, overallscore, goal1, goal2, goal3, goal4, goal5, goal6, goal7, goal8, goal9, goal10, goal11, goal12, goal13, goal15, goal16, goal17))

For question 3, we create 3 distinct databases according to the different type of event that we wwill analyse: disasters, COVID19 and conflicts. For the disasters, we only keep the years until 2021, because after this date, we don’t have data. For the conflicts, we only keep the years until 2016, because after this date, we don’t have data.

# Disasters
data_question3_1 <- all_Merge %>% filter(year<=2021) %>% select(c(code, year, country, continent, region, overallscore, goal1, goal2, goal3, goal4, goal5, goal6, goal7, goal8, goal9, goal10, goal11, goal12, goal13, goal15, goal16, goal7, total_deaths, no_injured, no_affected, no_homeless, total_affected, total_damages))

# COVID
data_question3_2 <- all_Merge %>% select(c(code, year, country, continent, region, overallscore, goal1, goal2, goal3, goal4, goal5, goal6, goal7, goal8, goal9, goal10, goal11, goal12, goal13, goal15, goal16, goal7, cases_per_million, deaths_per_million, stringency))

# Conflicts 
data_question3_3 <- all_Merge %>% filter(year<=2016) %>% select(c(code, year, country, continent, region, overallscore, goal1, goal2, goal3, goal4, goal5, goal6, goal7, goal8, goal9, goal10, goal11, goal12, goal13, goal15, goal16, goal7, ongoing, sum_deaths, pop_affected, area_affected, maxintensity))

Data for question 1

We create a column with the number of missing values by country over all the variables, except goal 1 and goal 10 that we already discussed. We decide to remove the countries that have more than 50 missing values.

see_missing1_1 <- data_question1 %>%
  group_by(code) %>%
  summarise(across(-c(X, year, country, continent, region, population, overallscore, goal1, goal2, goal3, goal4, goal5, goal6, goal7, goal8, goal9, goal10, goal11, goal12, goal13, goal15, goal16, goal17), 
                   ~ sum(is.na(.))) %>%
              mutate(num_missing = rowSums(across(everything()))) %>%
              filter(num_missing > 50))

data_question1 <- data_question1 %>% filter(!code %in% see_missing1_1$code)

list_country_deleted <- c(unique(see_missing1_1$code))

Here is the dataframe that allows us to see the countries that have missing values, how many and for which variables, when there are more than 50 in total.

code unemployment.rate GDPpercapita MilitaryExpenditurePercentGDP MiliratyExpenditurePercentGovExp internet_usage hf_score pf_law pf_security pf_movement pf_religion pf_assembly pf_expression pf_identity pf_score ef_government ef_legal ef_money ef_trade ef_regulation ef_score num_missing
BHS 0 0 21 21 0 0 14 0 0 0 0 0 0 0 0 0 0 0 0 0 56
BTN 0 0 21 21 0 13 0 0 0 0 0 0 0 13 0 0 13 13 10 13 117
COM 0 0 21 21 3 19 0 0 0 0 0 0 0 19 19 19 19 19 19 19 197
CPV 0 0 0 0 0 10 0 0 0 0 0 0 0 10 0 0 10 10 10 10 60
DJI 0 0 15 12 0 19 0 0 0 0 0 0 0 19 19 19 19 19 19 19 179
GIN 0 0 7 7 0 13 0 0 0 0 0 0 0 13 0 0 13 13 11 13 90
GMB 0 0 0 0 0 10 0 0 0 0 0 0 0 10 2 0 10 10 10 10 62
IRQ 0 0 4 4 2 16 0 0 0 0 0 0 0 16 3 0 16 16 16 16 109
KHM 0 0 0 0 3 10 0 0 0 0 0 0 0 10 0 0 10 10 10 10 63
LAO 0 0 7 7 0 14 0 0 0 0 0 0 0 14 0 0 14 14 13 14 97
LBN 0 0 0 0 0 10 0 0 0 0 0 0 0 10 0 0 10 10 10 10 60
LBR 0 0 0 0 2 14 0 0 0 0 0 0 0 14 0 0 14 14 10 14 82
QAT 0 0 12 12 0 10 0 0 0 0 0 0 0 10 7 0 10 10 10 10 91
SAU 0 0 0 0 0 10 0 0 0 0 0 0 0 10 0 0 10 10 10 10 60
SDN 0 0 5 5 9 16 0 0 0 0 0 0 0 16 0 0 16 16 16 16 115
SOM 0 13 21 13 4 19 0 0 0 0 0 0 0 19 19 19 19 19 19 19 203
SUR 0 0 21 21 0 10 0 0 0 0 0 0 10 10 5 0 10 10 10 10 117
SWZ 0 0 0 21 3 10 0 0 0 0 0 0 0 10 0 0 10 10 10 10 84
TJK 0 0 4 5 3 10 0 0 0 0 0 0 0 10 0 0 10 10 10 10 72
YEM 0 1 5 5 3 10 0 0 0 0 0 0 0 10 10 0 10 10 10 10 84

Now, looking at the remaining countries that have missing values and there number accross all variables, we decide to remove MilitaryExpenditurePercentGovExp, because it has too many missing values and it contains similar information to MilitaryExpenditurePercentGDP.

see_missing1_2 <- data_question1 %>%
  group_by(code) %>%
  summarise(across(-c(X, year, country, continent, region, population, overallscore, goal1, goal2, goal3, goal4, goal5, goal6, goal7, goal8, goal9, goal10, goal11, goal12, goal13, goal15, goal16, goal17),
                   ~ sum(is.na(.))) %>%
              mutate(num_missing = rowSums(across(everything()))) %>%
              filter(num_missing > 0))

data_question1 <- data_question1 %>% select(-MiliratyExpenditurePercentGovExp)

Here is the dataframe that allows us to see the countries that have missing values, how many and for which variables, after remoying the countries with more than 50.

code unemployment.rate GDPpercapita MilitaryExpenditurePercentGDP MiliratyExpenditurePercentGovExp internet_usage hf_score pf_law pf_security pf_movement pf_religion pf_assembly pf_expression pf_identity pf_score ef_government ef_legal ef_money ef_trade ef_regulation ef_score num_missing
AGO 0 0 0 0 0 5 0 0 0 0 0 0 0 5 0 0 5 5 5 5 30
ARE 0 0 6 6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 12
ARM 0 0 0 0 0 4 0 0 0 0 0 0 0 4 0 0 4 4 3 4 23
AUS 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3
AZE 0 0 0 0 2 4 0 0 0 0 0 0 0 4 0 0 4 4 0 4 22
BDI 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3
BFA 0 0 0 0 0 5 0 0 0 0 0 0 0 5 0 0 5 5 5 5 30
BIH 0 0 2 0 0 5 0 0 0 0 0 0 0 5 0 0 5 5 5 5 32
BLZ 0 0 0 0 3 0 13 0 0 0 0 0 0 0 0 0 0 0 0 0 16
BRB 0 0 21 21 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 45
CAF 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3
CIV 0 0 0 21 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 22
COD 0 0 0 21 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 21
COG 0 0 6 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 9
CRI 0 0 21 21 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 42
ETH 0 0 0 0 0 5 0 0 0 0 0 0 0 5 0 0 5 5 5 5 30
FJI 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2
GEO 0 0 0 0 0 2 0 0 0 0 0 0 0 2 0 0 3 2 0 2 11
GUY 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3
HTI 0 0 13 13 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 26
ISL 0 0 21 21 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 42
JAM 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2
JOR 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2
KAZ 0 0 0 0 0 5 0 0 0 0 0 0 0 5 0 0 5 5 5 5 30
KGZ 0 0 0 0 0 5 0 0 0 0 0 0 0 5 1 0 5 5 5 5 31
LKA 0 0 0 0 6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 6
LSO 0 0 0 0 0 5 0 0 0 0 0 0 0 5 0 0 5 5 5 5 30
MDA 0 0 0 0 3 5 0 0 0 0 0 0 0 5 0 0 5 5 5 5 33
MDG 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2
MKD 0 0 0 0 0 3 0 0 0 0 0 0 0 3 0 0 3 3 0 3 15
MMR 0 0 6 6 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 13
MNE 0 0 0 0 4 5 0 0 0 0 0 0 0 5 0 0 5 5 5 5 34
MNG 0 0 0 0 4 4 0 0 0 0 0 0 0 4 0 0 4 4 0 4 24
MOZ 0 0 0 0 0 3 0 0 0 0 0 0 0 3 0 0 3 3 0 3 15
MRT 0 0 0 7 0 5 0 0 0 0 0 0 0 5 0 0 5 5 5 5 37
MWI 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1
NER 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3
PAK 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1
PAN 0 0 21 21 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 42
PNG 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3
RWA 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 2
SRB 0 0 0 0 4 5 0 0 0 0 0 0 0 5 2 0 5 5 5 5 36
SYR 0 0 10 10 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 20
TCD 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2
TGO 0 0 5 5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 10
TTO 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3
USA 0 0 0 21 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 21
VEN 0 5 0 21 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 29
VNM 0 0 5 5 0 2 0 0 0 0 0 0 0 2 0 0 3 2 0 2 21
ZWE 0 0 3 8 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 11

GDP per capita

Only Venezuela has missing values that we can not fill, so we delete the country.

question1_missing_GDP <- data_question1 %>%
  group_by(code) %>%
  summarize(NaGDPpercapita = mean(is.na(GDPpercapita)))%>%
  filter(NaGDPpercapita != 0)

data_question1 <- data_question1 %>% filter(code!="VEN")

list_country_deleted <- c(list_country_deleted, "VEN")

Military expenditure in % of GDP

To begin with, we delete the countries with more than 30% missing values.

question1_missing_Military <- data_question1 %>%
  group_by(code) %>%
  summarize(NaMilitary = mean(is.na(MilitaryExpenditurePercentGDP)))%>%
  filter(NaMilitary != 0)

data_question1 <- data_question1 %>% filter(code!="BRB" & code!="CRI" & code!="HTI" & code!="ISL" & code!="PAN" & code!="SYR") 

list_country_deleted <- c(list_country_deleted, "BRB", "CRI", "HTI", "ISL", "PAN", "SYR") 

Then, we look at the distribution of the variable per region. Seeing that all are skewed distributions, we decide to replace the missing values, where there are less than 30% missing using the median by region.

question1_missing_Military <- data_question1 %>%
  group_by(code) %>%
  mutate(PercentageMissing = mean(is.na(MilitaryExpenditurePercentGDP))) %>% # Column % NAs
  ungroup() %>%
  group_by(region) %>%
  filter(sum(PercentageMissing, na.rm = TRUE) > 0)

Freq_Missing_Military <- ggplot(data = question1_missing_Military) +
  geom_histogram(aes(x = MilitaryExpenditurePercentGDP, 
                     fill = cut(PercentageMissing,
                                breaks = c(0, 0.1, 0.2, 0.3, 1),
                                labels = c("0-10%", "10-20%", "20-30%", "30-100%"))),
                 bins = 30) +
  labs(title = "Distribution of Military expenditures in % of GDP", x = "Military expenditures in % of GDP", y = "Frequency") +
  scale_fill_manual(values = c("0-10%" = "blue", "10-20%" = "green", "20-30%"="red","30-100%" = "black"), labels = c("0-10%", "10-20%", "20-30%","30-100%")) +
  guides(fill = guide_legend(title = "% missings")) +
  facet_wrap(~ region, nrow = 3)

print(Freq_Missing_Military)

data_question1 <- data_question1 %>%
  group_by(code) %>%
  mutate(
    PercentageMissingByCode = mean(is.na(MilitaryExpenditurePercentGDP))
  ) %>%
  ungroup() %>%  
  group_by(region) %>%
  mutate(
    MedianByRegion = median(MilitaryExpenditurePercentGDP, na.rm = TRUE),
    MilitaryExpenditurePercentGDP = ifelse(
      PercentageMissingByCode < 0.3 & !is.na(MilitaryExpenditurePercentGDP),
      MilitaryExpenditurePercentGDP,
      ifelse(PercentageMissingByCode < 0.3, MedianByRegion, MilitaryExpenditurePercentGDP)
    )
  ) %>%
  select(-PercentageMissingByCode, -MedianByRegion)

Internet usage

There are only low percentage of missing values.

question1_missing_Internet <- data_question1 %>%
  group_by(code) %>%
  summarize(NaInternet = mean(is.na(internet_usage)))%>%
  filter(NaInternet != 0)

We look at the evolution of the variable over time. We fill the missing values with linear interpolation, because all evolutions are in an increasing way and are almost straight lines, except for CIV that we delete.

question1_missing_Internet <- data_question1 %>%
  group_by(code) %>%
  mutate(PercentageMissing = mean(is.na(internet_usage))) %>% # Column % NAs
  filter(code %in% question1_missing_Internet$code)

Evol_Missing_Internet <- ggplot(data = question1_missing_Internet) +
  geom_point(aes(x = year, y = internet_usage, 
                 color = cut(PercentageMissing,
                             breaks = c(0, 0.1, 0.2, 0.3, 1),
                             labels = c("0-10%", "10-20%", "20-30%", "30-100%")))) +
  labs(title = "Evolution of internet usage over time", x = "Years from 2000 to 2022", y = "Internet usage") +
  scale_color_manual(values = c("0-10%" = "blue", "10-20%" = "green", "20-30%" = "red", "30-100%" = "black"),
                     labels = c("0-10%", "10-20%", "20-30%", "50-100%")) +
  guides(color = guide_legend(title = "% missings")) +
  scale_x_continuous(breaks=NULL)+
  facet_wrap(~ code, nrow = 4)

print(Evol_Missing_Internet)

list_code <- setdiff(unique(question1_missing_Internet$code), "CIV")
for (i in list_code) {
  country_data <- data_question1 %>% filter(code == i)
  interpolated_data <- na.interp(country_data$internet_usage)
  data_question1[data_question1$code == i, "internet_usage"] <- interpolated_data
}

data_question1 <- data_question1 %>% filter(code!="CIV")

list_country_deleted <- c(list_country_deleted, "CIV") 

Human freedom index

First, we remove hf_score, pf_score and ef_score, because there are many missing values and since these variables summarize the other ones, deleting the will not make us loose information.

data_question1 <- data_question1 %>% select(-c(hf_score, pf_score, ef_score))

Human freedom index

The variable pf_law has (many) NAs, but only for one country: BLZ, so we decide to remove it.

data_question1 <- data_question1 %>% filter(code!="BLZ")

list_country_deleted <- c(list_country_deleted, "BLZ") 

Economic freedom: government

Only KGZ and SRB have missing values, we plot the values over time and fill in the missing values by the year before, since there are only one and two missing values.

Evol_Missing_ef_gov <- data_question1 %>% group_by(code) %>% filter(code=="KGZ")
ggplot(Evol_Missing_ef_gov, aes(x = year, y = ef_government)) +
  geom_point() +
  labs(title = "Evolution of economic freedom: government over time in KGZ", x = "Years", y = "ef_gov")

Evol_Missing_ef_gov <- data_question1 %>% group_by(code) %>% filter(code=="SRB")
ggplot(Evol_Missing_ef_gov, aes(x = year, y = ef_government)) +
  geom_point() +
  labs(title = "Evolution of economic freedom: government over time in SRB", x = "Years", y = "ef_gov")

data_question1 <- data_question1 %>%
  mutate(ef_government = ifelse(code == "KGZ" & year == 2000 & is.na(ef_government), ef_government[which(code == "KGZ" & year == 2001)], ef_government))
data_question1 <- data_question1 %>%
  mutate(ef_government = ifelse(code == "SRB" & year == 2000 & is.na(ef_government), ef_government[which(code == "SRB" & year == 2002)], ef_government))
data_question1 <- data_question1 %>%
  mutate(ef_government = ifelse(code == "SRB" & year == 2001 & is.na(ef_government), ef_government[which(code == "SRB" & year == 2002)], ef_government))

Economic freedom: money

18 countries have missing values, but the percentage of missing values is always below 25%.

question1_missing_ef_money <- data_question1 %>%
  group_by(code) %>%
  summarize(Na_ef_money = mean(is.na(ef_money)))%>%
  filter(Na_ef_money != 0)

We look at the evolution of the variable over time. For the countries where this evolution is linear, we fill in the missing values using linear interpolation.

question1_missing_ef_money <- data_question1 %>%
  group_by(code) %>%
  mutate(PercentageMissing = mean(is.na(ef_money))) %>% # Column % NAs
  filter(code %in% question1_missing_ef_money$code)

Evol_Missing_ef_money <- ggplot(data = question1_missing_ef_money) +
  geom_point(aes(x = year, y = ef_money, 
                 color = cut(PercentageMissing,
                             breaks = c(0, 0.1, 0.2, 0.3, 1),
                             labels = c("0-10%", "10-20%", "20-30%", "30-100%")))) +
  labs(title = "Evolution of economiv freedom: money over time", x = "Years from 2000 to 2022", y = "ef_money") +
  scale_color_manual(values = c("0-10%" = "blue", "10-20%" = "green", "20-30%" = "red", "30-100%" = "black"),
                     labels = c("0-10%", "10-20%", "20-30%", "50-100%")) +
  guides(color = guide_legend(title = "% missings")) +
  facet_wrap(~ code, nrow = 4) +
  scale_x_continuous(breaks = NULL)

print(Evol_Missing_ef_money)

list_code <- c("ARM", "BFA", "BIH", "GEO", "KAZ", "LSO", "MDA", "MKD")
for (i in list_code) {
  country_data <- data_question1 %>% filter(code == i)
  interpolated_data <- na.interp(country_data$ef_money)
  data_question1[data_question1$code == i, "ef_money"] <- interpolated_data
}

Then, we look at the distribution of the variable per region. Seeing that all are skewed distributions, we decide to replace the missing values using the median by region.

question1_missing_ef_money <- data_question1 %>%
  group_by(code) %>%
  mutate(PercentageMissing = mean(is.na(ef_money))) %>% # Column % NAs
  ungroup() %>%
  group_by(region) %>%
  filter(sum(PercentageMissing, na.rm = TRUE) > 0)

Freq_Missing_ef_money <- ggplot(data = question1_missing_ef_money) +
  geom_histogram(aes(x = ef_money, 
                     fill = cut(PercentageMissing,
                                breaks = c(0, 0.1, 0.2, 0.3, 1),
                                labels = c("0-10%", "10-20%", "20-30%", "30-100%"))),
                 bins = 30) +
  labs(title = "Distribution of economic freedom: money", x = "ef_money", y = "Frequency") +
  scale_fill_manual(values = c("0-10%" = "blue", "10-20%" = "green", "20-30%"="red","30-100%" = "black"), labels = c("0-10%", "10-20%", "20-30%","30-100%")) +
  guides(fill = guide_legend(title = "% missings")) +
  facet_wrap(~ region, nrow = 3)

print(Freq_Missing_ef_money)

data_question1 <- data_question1 %>%
  group_by(code) %>%
  mutate(
    PercentageMissingByCode = mean(is.na(ef_money))
  ) %>%
  ungroup() %>% 
  group_by(region) %>%
  mutate(
    MedianByRegion = median(ef_money, na.rm = TRUE),
    ef_money = ifelse(
      PercentageMissingByCode < 0.3 & !is.na(ef_money),
      ef_money,
      ifelse(PercentageMissingByCode < 0.3, MedianByRegion, ef_money)
    )
  ) %>%
  select(-PercentageMissingByCode, -MedianByRegion)

Economic freedom: trade

19 countries have missing values, but the percentage of missing values is always below 25%.

question1_missing_ef_trade <- data_question1 %>%
  group_by(code) %>%
  summarize(Na_ef_trade = mean(is.na(ef_trade)))%>% # Column % NAs
  filter(Na_ef_trade != 0)

question1_missing_ef_trade <- data_question1 %>%
  group_by(code) %>%
  mutate(PercentageMissing = mean(is.na(ef_trade))) %>%
  filter(code %in% question1_missing_ef_trade$code)

We look at the evolution of the variable over time. For the countries where this evolution is linear, we fill in the missing values using linear interpolation.

Evol_Missing_ef_trade <- ggplot(data = question1_missing_ef_trade) +
  geom_point(aes(x = year, y = ef_trade, 
                 color = cut(PercentageMissing,
                             breaks = c(0, 0.1, 0.2, 0.3, 1),
                             labels = c("0-10%", "10-20%", "20-30%", "30-100%")))) +
  labs(title = "Evolution of economic freedom: trade over time", x = "Years from 2000 to 2022", y = "ef_trade") +
  scale_color_manual(values = c("0-10%" = "blue", "10-20%" = "green", "20-30%" = "red", "30-100%" = "black"),
                     labels = c("0-10%", "10-20%", "20-30%", "50-100%")) +
  guides(color = guide_legend(title = "% missings")) +
  facet_wrap(~ code, nrow = 4) +
  scale_x_continuous(breaks = NULL)

print(Evol_Missing_ef_trade)

# Linear interpolation for "AZE", "BFA", "ETH", "GEO", "VNH"
list_code <- c("AZE", "BFA", "ETH", "GEO", "VNH")
for (i in list_code) {
  country_data <- data_question1 %>% filter(code == i)
  interpolated_data <- na.interp(country_data$ef_trade)
  data_question1[data_question1$code == i, "ef_trade"] <- interpolated_data
}

Then, we look at the distribution of the variable per region. Seeing that all are skewed distributions, we decide to replace the missing values using the median by region.

question1_missing_ef_trade <- data_question1 %>%
  group_by(code) %>%
  mutate(PercentageMissing = mean(is.na(ef_trade))) %>% # Column % NAs
  ungroup() %>%
  group_by(region) %>%
  filter(sum(PercentageMissing, na.rm = TRUE) > 0)

Freq_Missing_ef_trade <- ggplot(data = question1_missing_ef_trade) +
  geom_histogram(aes(x = ef_trade, 
                     fill = cut(PercentageMissing,
                                breaks = c(0, 0.1, 0.2, 0.3, 1),
                                labels = c("0-10%", "10-20%", "20-30%", "30-100%"))),
                 bins = 30) +
  labs(title = "Distribution of economic freedom: trade", x = "ef_trade", y = "Frequency") +
  scale_fill_manual(values = c("0-10%" = "blue", "10-20%" = "green", "20-30%"="red","30-100%" = "black"), labels = c("0-10%", "10-20%", "20-30%","30-100%")) +
  guides(fill = guide_legend(title = "% missings")) +
  facet_wrap(~ region, nrow = 3)

print(Freq_Missing_ef_trade)

data_question1 <- data_question1 %>%
  group_by(code) %>%
  mutate(
    PercentageMissingByCode = mean(is.na(ef_trade))
  ) %>%
  ungroup() %>% 
  group_by(region) %>%
  mutate(
    MedianByRegion = median(ef_trade, na.rm = TRUE),
    ef_trade = ifelse(
      PercentageMissingByCode < 0.3 & !is.na(ef_trade),
      ef_trade,
      ifelse(PercentageMissingByCode < 0.3, MedianByRegion, ef_trade)
    )
  ) %>%
  select(-PercentageMissingByCode, -MedianByRegion)

Economic freedom: regulation

12 countries have missing values, but the percentage of missing values is always below 25%.

question1_missing_ef_regulation <- data_question1 %>%
  group_by(code) %>%
  summarize(Na_ef_regulation = mean(is.na(ef_regulation)))%>% # Column % NAs
  filter(Na_ef_regulation != 0)

We look at the evolution of the variable over time. For the countries where this evolution is linear, we fill in the missing values using linear interpolation.

question1_missing_ef_regulation <- data_question1 %>%
  group_by(code) %>%
  mutate(PercentageMissing = mean(is.na(ef_regulation))) %>%
  filter(code %in% question1_missing_ef_regulation$code)

Evol_Missing_ef_regulation <- ggplot(data = question1_missing_ef_regulation) +
  geom_point(aes(x = year, y = ef_regulation, 
                 color = cut(PercentageMissing,
                             breaks = c(0, 0.1, 0.2, 0.3, 1),
                             labels = c("0-10%", "10-20%", "20-30%", "30-100%")))) +
  labs(title = "Evolution of economic freedom: regulation over time", x = "Years from 2000 to 2022", y = "ef_regulation") +
  scale_color_manual(values = c("0-10%" = "blue", "10-20%" = "green", "20-30%" = "red", "30-100%" = "black"),
                     labels = c("0-10%", "10-20%", "20-30%", "50-100%")) +
  guides(color = guide_legend(title = "% missings")) +
  facet_wrap(~ code, nrow = 4)

print(Evol_Missing_ef_regulation)

list_code <- c("ETH", "KAZ", "MDA", "SRB")
for (i in list_code) {
  country_data <- data_question1 %>% filter(code == i)
  interpolated_data <- na.interp(country_data$ef_regulation)
  data_question1[data_question1$code == i, "ef_regulation"] <- interpolated_data
}

Then, we look at the distribution of the variable per region. Seeing that all are skewed distributions, we decide to replace the missing values using the median by region.

question1_missing_ef_regulation <- data_question1 %>%
  group_by(code) %>%
  mutate(PercentageMissing = mean(is.na(ef_regulation))) %>% # Column % NAs
  ungroup() %>%
  group_by(region) %>%
  filter(sum(PercentageMissing, na.rm = TRUE) > 0)

Freq_Missing_ef_regulation <- ggplot(data = question1_missing_ef_regulation) +
  geom_histogram(aes(x = ef_regulation, 
                     fill = cut(PercentageMissing,
                                breaks = c(0, 0.1, 0.2, 0.3, 1),
                                labels = c("0-10%", "10-20%", "20-30%", "30-100%"))),
                 bins = 100) +
  labs(title = "Distribution of economic freedom: regulation", x = "ef_regulation", y = "Frequency") +
  scale_fill_manual(values = c("0-10%" = "blue", "10-20%" = "green", "20-30%"="red","30-100%" = "black"), labels = c("0-10%", "10-20%", "20-30%","30-100%")) +
  guides(fill = guide_legend(title = "% missings")) +
  facet_wrap(~ region, nrow = 3)

print(Freq_Missing_ef_regulation)

data_question1 <- data_question1 %>%
  group_by(code) %>%
  mutate(
    PercentageMissingByCode = mean(is.na(ef_regulation))
  ) %>%
  ungroup() %>% 
  group_by(region) %>%
  mutate(
    MedianByRegion = median(ef_regulation, na.rm = TRUE),
    ef_regulation = ifelse(
      PercentageMissingByCode < 0.3 & !is.na(ef_regulation),
      ef_regulation,
      ifelse(PercentageMissingByCode < 0.3, MedianByRegion, ef_regulation)
    )
  ) %>%
  select(-PercentageMissingByCode, -MedianByRegion) %>%
  ungroup()

na_count <- sapply(data_question1, function(x) sum(is.na(x)))
print(na_count)
#>                             X                          code 
#>                             0                             0 
#>                          year                       country 
#>                             0                             0 
#>                     continent                        region 
#>                             0                             0 
#>                    population                  overallscore 
#>                             0                             0 
#>                         goal1                         goal2 
#>                           105                             0 
#>                         goal3                         goal4 
#>                             0                             0 
#>                         goal5                         goal6 
#>                             0                             0 
#>                         goal7                         goal8 
#>                             0                             0 
#>                         goal9                        goal10 
#>                             0                           105 
#>                        goal11                        goal12 
#>                             0                             0 
#>                        goal13                        goal15 
#>                             0                             0 
#>                        goal16                        goal17 
#>                             0                             0 
#>             unemployment.rate                  GDPpercapita 
#>                             0                             0 
#> MilitaryExpenditurePercentGDP                internet_usage 
#>                             0                             0 
#>                        pf_law                   pf_security 
#>                             0                             0 
#>                   pf_movement                   pf_religion 
#>                             0                             0 
#>                   pf_assembly                 pf_expression 
#>                             0                             0 
#>                   pf_identity                 ef_government 
#>                             0                             0 
#>                      ef_legal                      ef_money 
#>                             0                             0 
#>                      ef_trade                 ef_regulation 
#>                             0                             0

Now, we notice that there were only missing values for goals 1 and 10. As we did before, we have started to investigate where are located the NAs in our dataset for first goal1, then goal 10.

# goal1
question1_missing_goal1 <- data_question1 %>%
  group_by(code) %>%
  summarize(Na_goal1 = mean(is.na(goal1)))%>%
  filter(Na_goal1 != 0)

data_question1 <- data_question1 %>% filter(!code %in% question1_missing_goal1$code)

# Update List of countries deleted
list_country_deleted <- c(list_country_deleted, "KWT","NZL","OMN","SGP","UKR")

# still 42 NA values goal10

We had found that the missing values were located in only 5 countries. So we have decided to get rid of them. At this stage, there were only 42 remaining missing values. Then, same step for goal 10.

#goal10
question1_missing_goal10 <- data_question1 %>%
  group_by(code) %>%
  summarize(Na_goal10 = mean(is.na(goal10)))%>%
  filter(Na_goal10 != 0)

data_question1 <- data_question1 %>% filter(!code %in% question1_missing_goal10$code)

# Update List of countries deleted
list_country_deleted <- c(list_country_deleted, "GUY","TTO")

We have found the 2 lasts contries containing missing values. Now, our dataset is completely clean and ready to be used for our question 1.

Data for question 2 and 4

We create a column with the number of missing values by country over all the variables, except goal 1 and goal 10 that we already discussed. Since there are no other missing values, we stop here.

see_missing24 <- data_question24 %>%
  group_by(code) %>%
  summarise(across(everything(), ~ sum(is.na(.))) %>%
              mutate(num_missing = rowSums(across(everything()))) %>%
              filter(num_missing > 0))

Data for question 3

We create a column with the number of missing values by country over all the variables, except goal 1 and goal 10 that we already discussed. Since there are no other missing values, we stop here.

Disasters

We create a column with the number of missing values by country over all the variables, except goal 1 and goal 10 that we already discussed. We find out that there are many missing values

see_missing3_1 <- data_question3_1 %>%
  group_by(code) %>%
  summarise(across(-c(goal1, goal10),  # Exclude columns "goal1" and "goal10"
                   ~ sum(is.na(.))) %>%
              mutate(num_missing = rowSums(across(everything()))) %>%
              filter(num_missing > 0))
kable(see_missing3_1)
code year country continent region overallscore goal2 goal3 goal4 goal5 goal6 goal7 goal8 goal9 goal11 goal12 goal13 goal15 goal16 total_deaths no_injured no_affected no_homeless total_affected total_damages num_missing
AGO 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 6
ALB 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 9 9 9 9 9 9 54
ARE 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 21 21 21 21 21 21 126
ARM 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 15 15 15 15 15 15 90
AUT 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 8 8 8 8 8 8 48
AZE 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 17 17 17 17 17 17 102
BDI 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 3 3 3 3 3 18
BEL 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 5 5 5 5 5 5 30
BEN 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 7 7 7 7 7 7 42
BFA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 5 5 5 5 5 5 30
BGR 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 9 9 9 9 9 9 54
BHS 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 10 10 10 10 10 10 60
BIH 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 9 9 9 9 9 9 54
BLZ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 13 13 13 13 13 13 78
BOL 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 6
BRB 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 16 16 16 16 16 16 96
BTN 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 17 17 17 17 17 17 102
BWA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 11 11 11 11 11 11 66
CAF 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 3 3 3 3 3 18
CHE 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4 4 4 4 4 4 24
CHL 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 3 3 3 3 3 18
CIV 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 7 7 7 7 7 7 42
CMR 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 3 3 3 3 3 18
COD 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 6
COG 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 8 8 8 8 8 8 48
COM 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 14 14 14 14 14 14 84
CPV 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 16 16 16 16 16 16 96
CRI 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 6
CYP 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 17 17 17 17 17 17 102
CZE 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 7 7 7 7 7 7 42
DEU 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 2 2 2 2 2 12
DJI 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 13 13 13 13 13 13 78
DNK 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 17 17 17 17 17 17 102
DOM 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 2 2 2 2 2 12
DZA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 3 3 3 3 3 18
ECU 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 5 5 5 5 5 5 30
EGY 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 14 14 14 14 14 14 84
ESP 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 6
EST 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 19 19 19 19 19 19 114
ETH 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4 4 4 4 4 4 24
FIN 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 21 21 21 21 21 21 126
FJI 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 5 5 5 5 5 5 30
GAB 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 16 16 16 16 16 16 96
GBR 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4 4 4 4 4 4 24
GEO 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 7 7 7 7 7 7 42
GHA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4 4 4 4 4 4 24
GIN 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 3 3 3 3 3 18
GMB 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 6 6 6 6 6 6 36
GRC 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 3 3 3 3 3 18
GTM 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 6
GUY 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 15 15 15 15 15 15 90
HND 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 2 2 2 2 2 12
HRV 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 8 8 8 8 8 8 48
HUN 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 5 5 5 5 5 5 30
IRL 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 11 11 11 11 11 11 66
IRN 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 6
IRQ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 7 7 7 7 7 7 42
ISL 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 20 20 20 20 20 20 120
ISR 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 15 15 15 15 15 15 90
JAM 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 9 9 9 9 9 9 54
JOR 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 19 19 19 19 19 19 114
KAZ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 7 7 7 7 7 7 42
KGZ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 6 6 6 6 6 6 36
KHM 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4 4 4 4 4 4 24
KOR 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 3 3 3 3 3 18
KWT 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 20 20 20 20 20 20 120
LAO 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 6 6 6 6 6 6 36
LBN 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 15 15 15 15 15 15 90
LBR 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 12 12 12 12 12 12 72
LSO 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 11 11 11 11 11 11 66
LTU 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 16 16 16 16 16 16 96
LUX 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 18 18 18 18 18 18 108
LVA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 16 16 16 16 16 16 96
MAR 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 5 5 5 5 5 5 30
MDA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 14 14 14 14 14 14 84
MDG 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 6
MKD 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 7 7 7 7 7 7 42
MLI 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 3 3 3 3 3 18
MMR 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 2 2 2 2 2 12
MNE 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 17 17 17 17 17 17 102
MNG 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 9 9 9 9 9 9 54
MOZ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 6
MRT 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 7 7 7 7 7 7 42
MUS 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 17 17 17 17 17 17 102
MWI 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 2 2 2 2 2 12
MYS 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 2 2 2 2 2 12
NAM 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4 4 4 4 4 4 24
NIC 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 3 3 3 3 3 18
NLD 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 8 8 8 8 8 8 48
NOR 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 15 15 15 15 15 15 90
NZL 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 3 3 3 3 3 18
OMN 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 10 10 10 10 10 10 60
PAN 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 6
PNG 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 3 3 3 3 3 18
POL 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 6
PRT 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 6 6 6 6 6 6 36
PRY 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 3 3 3 3 3 18
QAT 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 21 21 21 21 21 21 126
ROU 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 6
RUS 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 2 2 2 2 2 12
RWA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4 4 4 4 4 4 24
SAU 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 6 6 6 6 6 6 36
SEN 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 5 5 5 5 5 5 30
SGP 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 19 19 19 19 19 19 114
SLE 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 6 6 6 6 6 6 36
SLV 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 3 3 3 3 3 18
SRB 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 10 10 10 10 10 10 60
SUR 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 19 19 19 19 19 19 114
SVK 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 7 7 7 7 7 7 42
SVN 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 15 15 15 15 15 15 90
SWE 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 16 16 16 16 16 16 96
SWZ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 12 12 12 12 12 12 72
SYR 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 13 13 13 13 13 13 78
TCD 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 5 5 5 5 5 5 30
TGO 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 10 10 10 10 10 10 60
THA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 6
TTO 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 18 18 18 18 18 18 108
TUN 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 15 15 15 15 15 15 90
TUR 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 6
TZA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 2 2 2 2 2 12
UGA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 2 2 2 2 2 12
UKR 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 6 6 6 6 6 6 36
URY 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 7 7 7 7 7 7 42
VEN 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 5 5 5 5 5 5 30
YEM 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4 4 4 4 4 4 24
ZAF 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 6
ZMB 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 5 5 5 5 5 5 30
ZWE 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4 4 4 4 4 4 24

In this particular case, even if there are many missing values in our disaster dataset, we made the hypothesis that disaster events can not happen every year for every country given that these are uncontrollable and non-recurring events. Therefore the NAs that we encounter will become zeroes, implying that there have been no climatic disasters.

data_question3_1[is.na(data_question3_1)] <- 0

COVID19

We create a column with the number of missing values by country over all the variables, except goal 1 and goal 10 that we already discussed. Since there are no other missing values, we stop here.

see_missing3_2 <- data_question3_2 %>%
  group_by(code) %>%
  summarise(across(-c(goal1, goal10),  # Exclude columns "goal1" and "goal10"
                   ~ sum(is.na(.))) %>%
              mutate(num_missing = rowSums(across(everything()))) %>%
              filter(num_missing > 0))

Conflicts

We create a column with the number of missing values by country over all the variables, except goal 1 and goal 10 that we already discussed.Two countries have missing values, we remove them (MNE and SRB).

see_missing3_3 <- data_question3_3 %>%
  group_by(code) %>%
  summarise(across(-c(goal1, goal10),  # Exclude columns "goal1" and "goal10"
                   ~ sum(is.na(.))) %>%
              mutate(num_missing = rowSums(across(everything()))) %>%
              filter(num_missing > 0))

data_question3_3 <- data_question3_3 %>% filter(!code %in% c("MNE","SRB"))

##### EXPORT as CSV #####
write.csv(data_question1, file = here("scripts","data","data_question1.csv"))
write.csv(data_question24, file = here("scripts","data","data_question24.csv"))
write.csv(data_question3_1, file = here("scripts","data","data_question3_1.csv"))
write.csv(data_question3_2, file = here("scripts","data","data_question3_2.csv"))
write.csv(data_question3_3, file = here("scripts","data","data_question3_3.csv"))